Static Binary Analysis And Transformation For Sandboxing Untrusted Plugins

of the Thesis Static Binary Analysis And Transformation For Sandboxing Untrusted Plugins by Prateek Saxena Master of Science in Computer Science Stony Brook University 2007 Computers today have become a integral part of daily activities for users who rely on them as means of communication, financial management, entertainment, and business. Moreover, users today are depending increasingly on off-the-shelf software from untrusted sources like the Internet for everyday life. This factor has prompted research on mitigating the threat of untrusted programs running on the end user’s personal computers. Relatively lesser focus has been laid on the threat of software extensions such as plug-ins and modules for trusted host system such as web browsers and email clients. With the alarming increase in malware that continues to defeat state-of-the-art defenses and evades detection, this work aims to be part of a general body of “proactive” defense technique that gives strong guarantees against future attacks, as opposed to a defense strategy that is developed in response to known vulnerabilities and their exploits. In this context, this thesis has three objectives. First, it analyzes the threat model imposed by shared-address space extensions and modules that plug into larger host systems. Second, it surveys the limitations of existing mechanisms to deal with this threat, and proposes a practical approach to confinement of untrusted extensions. Finally, it presents a robust static binary rewriting and analysis framework that has more general applicability as tool for analysis and instrumentation for security applications where no source code is available.